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ABSTRACT 



A major aim of the social studies according to state and 
local educational goals is to foster students’ thinking skills. This study 
was conducted to determine, according to Bloom's taxonomy, the levels of 
questioning generated by third-grade social studies textbooks. It examined 
the extent to which each selected textbook emphasized higher levels of 
questioning (above knowledge level) and compared the textbooks’ frequency 
ratings of knowledge- and above -knowledge -level questioning. The two series 
selected for study were: "Communities: Harcourt Brace Social Studies” (2000) 
and "Communities: Macmillan-McGraw Social Studies" (1997). Lesson plans and 
accompanying questions for in-class use were obtained for classification. 
Three raters were trained in question classification using the six major 
categories of Bloom's taxonomy (1956). Based on the assumption that the 
Harcourt Brace and Macmillan-McGraw textbooks are representative of all new 
social studies textbooks, an encouraging trend emerges from the study's 
analysis. These textbooks offer opportunities for applying and evaluating 
information in higher order quest ion -and -answer classroom recitation 
previously omitted in textbook publications. Findings suggest that elementary 
students exposed to the most recent editions of social studies textbooks will 
have opportunities, as encouraged by questions included in teachers' 
editions, to understand, apply, synthesize, and evaluate critically social 
studies concepts. Contains a table, 2 figures, and 19 references. (BT) 
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A major aim of the social studies according to state and local educational goals is 
to foster students’ thinking skills. Indeed, an examination of curricular materials including 
textbooks reveals that critical thinking is paramount among desired learning outcomes. 
Realization of this goal is doubtful, however, if elementary students are treated to a 
meager intellectual diet offering little more than mere memorization of social studies facts. 

Although the recurrent theme of social studies literature has accentuated the topic 
of higher level thinking, instructional practice has changed very little during the past 
century. Research has indicated that textbook-dominated instruction prevails and the 
major portion of allocated social studies time is spent reading and reciting information 
from the text (Woodward, 1993; Woodward & Elliot, 1990). Therefore, social studies 
instruction and ultimately a pupil’s train of thought most often parallel the cognitive 
level(s) of the textbook. 

A rapidly changing, technologically advanced world necessitates that students 
develop the ability to make adaptations, think creatively and critically, and solve complex 
problems. To reach the oft-stated goal of critical thinking in the social studies, the 
curriculum must afford its participants an opportunity to engage in problem-solving 
exercises. The role of questioning in developing critical thinking skills has been well 
documented in the literature. Taba, Levine, and Elzey (1964), in discussion of its 
importance, referred to questioning as “the most influential teaching act” (p. 53). 
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Indeed, the literature has substantiated the role of higher cognitive questioning in evoking 
critical thinking (Andre, 1979; Doak, 1970; Hunkins, 1970; Redfield & Rousseau, 1981; 
Taba et al., 1964). In this regard, teachers actually define the limits of student thought 
through their questioning strategies; thus, students who receive memory-only questions 
are never challenged to exercise critical thinking skills. 

Because teachers rely heavily on textbooks and the accompanying teachers’ 
editions as the primary source of social studies instruction (EPIE, 1979; Hertzberg, 1981; 
Weiss, 1978; Woodward, 1993; Woodward & Elliot, 1990), elementary social studies 
textbooks and their accompanying resource materials (i.e., teachers’ editions) must be 
analyzed to determine their capacity to promote development of higher level thinking 
skills. A central concern is: What levels of questioning are included in elementary social 
studies teachers’ manuals for classroom use? 

Over the past decade, two recurrent trends have emerged in the textbook 
marketplace. First, teachers’ manuals are extremely comprehensive, including numerous 
options for teachers to use with each existing chapter of social studies content. Although 
teachers’ editions have changed significantly in recent years to include a wealth of lesson 
plan options, one component remains unchanged: the preponderance of questions 
interspersed throughout the individual plans designed to parallel the text segment targeted 
for instruction and classroom discussion. Research has indicated that teachers continue to 
rely heavily on such lesson plans (Armbruster & Ostertag, 1989; Armbruster & Others, 
1990; SeminofF, 1990; Woodward, 1993). 
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Second, due to the competitive marketplace that exists among textbook publishers, 
each company has expanded the list of ancillary materials that accompany its textbook 
series. It is common for each publisher to provide supplementary materials such as study 
guides, chapter-end and unit tests, posters, maps, parent letters, and the like. Because of 
the social studies goals related to critical thinking cited at both the national and state 
levels, the competition necessitates publishers to preface their texts with impressive claims 
that their materials elicit higher order thinking skills. Indeed, an analysis of publications 
dated 1997 through 2000 claim the inclusion of “divergent and open-ended questions” as 
well as questions reflective of “Bloom’s higher levels.” Even though this renewed 
emphasis on critical thinking claimed by textbook publishers is refreshing, teachers must 
ask: Have the new social studies textbooks changed my approach to questioning to 
include critical thinking? 

As of this writing, no researcher has reported studying the cognitive levels of 
elementary social studies questioning that accompany the new (i.e., 1997-2000) teachers’ 
editions designed for recitation during classroom lessons. Identified research related to 
textbook questioning in the area of elementary social studies has proved to be quite dated 
(Davis & Hunkins, 1969; Marksberry, 1969) and to have focused on questions included in 
students’ books for use after reading a segment of text (i.e., end-of-chapter questions). 

Our purpose in this study was to determine, according to Bloom’s (1956) 
taxonomy, the levels of questioning generated by third-grade social studies textbooks. We 
also examined the extent to which each selected textbook emphasized higher levels of 
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questioning (above knowledge level) and compared the two textbooks frequency ratings 
of knowledge- and above-knowledge-level questioning. 

Method and Procedures 

Materials Selection 

Information regarding social studies textbook adoption was compiled from the 
state departments of education of Alabama, California, and Texas. This information was 
used to rank order the third-grade social studies textbook series used most frequently for 
these states. As a result, the two series selected for study were (a) Communities: 

Harcourt Brace Social Studies (2000), hereafter referred to as HBJ; and (b) Communities! 
Marmillan-McGraw Social Studies (1997), hereafter called Macmillan. 

Lesson plans and accompanying questions for in-class use included in the teachers’ 
editions of the HBJ and Macmillan social studies texts were obtained for classification. 
These lesson plan questions related to the text segment currently targeted for reading, 
recitation, and classroom instruction. For each textbook, 100 questions were randomly 
selected for analysis. 

Raters and Question Classification 

Three raters were trained in question classification using the six major categories 
of Bloom’s (1956) taxonomy: (a) knowledge, (b) comprehension, (c) application, (d) 
analysis, (e) synthesis, and (f) evaluation. Prior to the study, these raters obtained an 
interrater agreement of .98 on a sample of social studies questions not included in the 
actual classification analysis. For the study, the 200 textbook questions were rated 
independently by the three raters, the results were compared, and the questions were 
O 

ERLC 



7 



Cognitive Levels of Questioning 7 



placed in the classification category ascribed by the majority of raters. Interrater 
agreement achieved during the actual classification study reached 94%. 

Results 

Table 1 shows the frequency distribution of the question classification for 100 
randomly selected questions in each of the two social studies textbooks. As indicated in 
Table 1, a total of 76 (38%) questions in the two textbooks were judged as knowledge- 
level questions (i.e., lower order, with emphasis on memorization of social studies facts) 



Insert Table 1 about here 



whereas 124 (62%) questions were rated as above-knowledge-level or higher level 
questions (i.e., higher order, with emphasis on understanding and/or applying social 
studies information). A chi-square goodness-of-fit comparison of knowledge- and above- 
knowledge-level questions yielded a significant (p < .001) chi-square value of 1 1.52, 
indicating that the two textbooks significantly emphasized higher order over lower order 
questioning (see Figure 1 for graphical representation of this finding). A chi-square test 



Insert Figure 1 about here 



for independence determined a significant difference (x 2 = 29.05, p < .001) existed 
between the two textbooks in the number of knowledge- and above-knowledge-level 
questions. This comparison indicates the HBJ third-grade social studies textbook 
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contained significantly more higher order questions than the third-grade Macmillan social 
studies textbook (see Figure 2 for graphical representation of this finding). 



Insert Figure 2 about here 
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Discussion 

Based on the assumption that the HBJ and Macmillan textbooks are representative 
of all new social studies textbooks, an encouraging trend emerges from the analysis 
conducted in this study: These textbooks offer opportunities for applying and evaluating 
information in higher order question-and-answer classroom recitation previously omitted 
in textbook publications (Sewall & Emberling, 1998; Seminoff, 1981), indicating a new 
emphasis on cognition rather than focusing on memorization and recall of isolated bits of 
factual information. Previous literature has consistently identified a gap between the 
stated goals of elementary social studies and actual classroom practices, particularly in the 
emphasis textbooks have placed on higher order thinking (e.g.. Woodward, 1993). The 
evidence presented in this study suggests that elementary students exposed to the most 
recent editions of social studies textbooks will have opportunities, as encouraged by 
questions included in teachers’ editions, to understand, apply, synthesize, and evaluate 
critically social studies concepts. Indeed, research has indicated that questions from 
teachers’ editions in elementary social studies texts are used extensively for classroom 
recitation (Woodward, 1993; Woodward & Elliot, 1990; Hertzberg, 1981) if exposing 
students to critical thinkin g skills through a balance of cognitive questions is an integral 
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part of the elementary social studies text, the HBJ and Macmillan textbooks meet that 
objective well. 

Although a statistically significant difference between the textbooks indicates that 
HBJ is superior to Macmillan in the area of higher level questioning, it is an important 
finding that Macmillan included a near-even split in its emphasis of knowledge- versus 
above-knowledge-level questioning. This cognitive balance in the levels of questioning is 
welcome information and should be applauded given the meager inclusion of higher level 
questions in previous publications. It should be remembered that questions included in 
teachers’ editions impact the quality and depth of concept discussion during an in-class 
lesson. If successful social studies is equated solely with an ability to memorize a given 
body of knowledge, as social studies textbooks historically have promoted, students are 
deprived of a challenging intellectual experience. As Taba et al. concluded as early as 
1964, “Teachers either enhance or limit student thought with their questioning” (p. 54). 
Further, teachers and curriculum specialists must consider this and other studies of 
questioning levels with regard to the fundamental goals of social studies, which are 
knowledge, skills, values, and social participation. For example, no amount of 
memorization can foster an adequate development of map and globe skills; students must 
be actively involved in using these skills in various situations to attain mastery. Similarly, 
students must participate in group life in order to attain social process skills. Concept 
learning, which is a vital part of the social studies curriculum, cannot occur in a setting 
that promotes memorization of facts alone because committing a concept definition to 
memory will not ensure true learning. 
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The data obtained from the current research imply that students who respond to 
questions included in the teachers’ edition of the Macmillan textbook will receive balanced 
levels of questioning. More exciting, however, is the preponderance of higher level 
questions demonstrated by the HBJ textbook teachers’ edition. In order to answer the 
majority of questions in the HBJ textbook, students must demonstrate higher cognitive 
abilities such as application and evaluation. Having analyzed the cognitive emphasis of 
questioning in social studies textbooks in the early 1990s, and noting from the present 
study’s data consistent attention to questioning in the above-knowledge levels of Bloom’s 
(1956) taxonomy, we recognize an unparalleled shift both publishers’ effort to bridge the 
gap between stated goals and philosophies about textbooks and actual textbook content. 
We hope that publishers of future elementary social studies texts will follow and improve 
upon the existing models that were examined in this research. 
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Table 1 



Taxonomy 


Category 


Questions 


Total 


HBJ 


Macmillan 


Number 


% 


Knowledge level 


19 


57 


76 


38.0 


Knowledge 


19 


57 


76 


38.0 


Above knowledge level 


81 


43 


124 


62.0 


Comprehension 


17 


15 


32 


16.0 


Application 


25 


6 


31 


15.5 


Analysis 


11 


8 


19 


9.5 


Synthesis 


8 


1 


9 


4.5 


Evaluation 


20 


13 


33 


16.5 


Total 


100 


100 


200 


100.0 
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Figure Caption 

Figure 1 . Significant (p < .001) emphasis demonstrated by HBJ and Macmillan textbooks 
(combined) on above-knowledge-level as opposed to knowledge-level questioning. 
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Figure 1. Significant (p < .001) emphasis demonstrated 
by HBJ and Macmillan textbooks (combined) on above- 
knowledge-level as opposed to knowledge-level questioning. 
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Figure Caption 

Figure 2. Significant (p < .001) emphasis demonstrated by the HBJ over the Macmillan 
textbook on above-knowledge-level as opposed to knowledge-level questioning. 
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Figure 2. Significant (p < .001) emphasis demonstrated by 
the HBJ over the Macmillan textbook on above-knowledge- 
level as opposed to knowledge-level questioning. 
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